Interplay between superconductivity and flux phase in the t-J model 
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^\ , We study the phase diagram of the t-J model using a mean field type approximation within the 

^\ ' Baym-KadanofF perturbation expansion for Hubbard X-operators. The line separating the normal 

T-H I state from a d-wave flux or bond-order state starts near optimal doping at T = and rises quickly 

^jQ. with decreasing doping. The transition temperature Tc for d-wave superconductivity increases mono- 

' tonically in the overdoped region towards optimal doping. Near optimal doping a strong competition 

, between the two d-wave order parameters sets in leading to a strong suppression of Tc in the under- 

^ ■ doped region. Treating for simplicity the flux phase as commensurate the superconducting and flux 

OO ' phases coexist in the underdoped region below Tc, whereas a pure flux phase exists above Tc with 

, a pseudo-gap of d-wave symmetry in the excitation spectrum. We also find that incommensurate 

■ charge-density-wave ground states due to Coulomb interactions do not modify strongly the above 
f-H I phase diagram near the superconducting phase, at least, as long as the latter exists at all. 

Q ! PACS numbers: 74.20.Mn, 74.25. Dw., 71.27.-|-a 

; 
u • 

PH- I. INTRODUCTION 

^ : 

^ ; ' It is widely accepted that the mechanism causing high- Tc superconductivity in the cuprates is intimately related to 
, the underlying properties of the normal state. However, there exists presently no agreement on what an appropriate 
' and correct description of the normal state is. One reason for this is that experiments indicate the presence of a wh 

1 \ variety of important fluctuations in the normal state. At low dopings antiferromagnetic fluctuations dominatetl 

■ The associated quantum critical point, however, lies far apart from optimal doping so that the relevance of magnetic 
^ ' fluctuations for high- Tc superconductivity is not evident. Other fluctuations which haye^ been ccm^ered to be 
^ relevant in thc,iiormal state are associated with resonance-valence bondsBlJ, flux phasesU^Q, stripesEHlEil, or charge 

density waveal3ll3. The close proximity of high- Tc superconductivity and structural phases or cross-overs also follows 
from thet-gxjperimental observation of a spin-gap or pseudo-gap of d-wave symmetry in the underdoped regime of the 
' cuprateso Ea. It is thus desirable to study in more detail the phase diagram of models relevant to the cuprates and 
^ to identify instabilities, their symmetries and their interactions. P|— , 

The low energy physics of the Cu02 layers of the cuprates is well described by the t-J modelclo. In order to 
^vq ' be able to carry out systematic approximations the two spin degrees of freedom per site are often increased to N 
00 degrees of freedom so that 1/iV can be used as a small parameteit3£j. For our purposes it is convenient to consider 
two spin degrees of freedom as in the original model and to extend the number of orbitals per site from one to N/2. 
00 In this way the symmetry group of the Hamiltonian of the t — J model is enlarged from SU (2) to the symplectic 
group Sp{N-l2-). Based on liALexpansions it has been shown that this model possesses at large TV's two intrinsic 
instaJpUitiesCJ. The first onecj E£l is associated with d-wave superconductivity and is obtained in 0{1/N). The second 
onec3"E3 is related to a flux or bond-order wave state of d-wave symmetry and is obtained already in the leading order 
C 0(1). The instability towards superconductivity is found for all dopings; the corresponding transition temperature 
Tc increases from the overdoped side with decreasing doping. Near and below optimal doping the second instability 
(~| sets in. It occurs in the d-wave channel for the relevant region of parameters. Both its location and its symmetry fits 
O • well to the experimental observations of a d-wave pseudogap in the underdoped region. It is thus of high interest to 
O ■ study these two intrinsic instabilities of the t — J model in more detail and this is the subject of this paper. 
^ ' One obstacle for such a study is the fact that the two instabilities occur in different orders of the expansion. 
Whereas the transition between the normal and charge-density wave or flux states can be obtained by comparing 
0(1) contributions to the free energy the superconducting part is of 0(1/A^). Nevertheless, the calculation of the 
i superconducting phase boundary is unique and involves only Green's functions of leading order in the 1/N expansion. 
■ " ' The position of the boundary in the phase diagram, on the other hand, depends on the value of N. We will use the 
1/A^ expansion in the sense that it allows to select a set of diagrams which are the leading ones at least at large A^'s. 
After this selection has been made we put N = 2. The anomalous self-energy of 0{1/N) contains quite a number of 
terms. It has recently been shown that only the d-wave contributions are important and that for this symmetry the 
retarded terms cancel each other to a large extent. To simplify our treatment we will thus only keep instantaneous 
terms in the d-wave channel for the superconducting part. 

In the following we will enforce the constraints of the t — J model by using A-operatorstilcjCj. Such a formulation 
is not more involved than the more familiar one using a slave boson formulation. It has, however, the advantage that 
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all quantities such as the order parameters are gauge invariant in the sense of slave boson theory. The outline of the 
paper is the following: in section |l| the formalism of our approach is introduced, and in section III we discuss the 
instability of the normal state towards incommensurate and commensurate flux phases, clarifying the range of validity 
of treatments which only take commensurate flux phases into account. The interplay between superconductivity and 
flux phase is discussed in detail in section |^ Finally we investigate in section ^ the role of phase separation and the 
effect of Coulomb interaction on the phase diagram. 



II. MEAN-FIELD EQUATIONS IN TERMS OF X-OPERATORS 

Using X-operators the Hamiltonian of our generalized t — J model has the form 

H = - y F —xfxf 

p=l.-.N p.q=l.-.N 

E Jjj vpp yqq I ^ij vpp yqq /-, \ 

ij ij 

p,g=l-..N p,g = l-.-N 

i and j run over the sites of a lattice. For N = 2 the operators Xf^ are identical with the projection operators 
I ^ >< ^ I , where | ^ > denotes for p = the empty state and for p — 1,2 singly occupied states at site i with spin up 
and down, and Jy are hopping and exchange constants, respectively, and both are assumed to act only between 
nearest neighbors. The last term in Eq.(l) describes the Coulomb repulsion between electrons. 

The extension from N = 2 degrees of freedom per site to a general N is accomplished by introducing a flavor index 
/i = 1, N /2 which enumerates N/2 copies of the original orbital. The index p is then a composite index p = {a, /i), 
where a denotes a spin index, and can be chosen to run from to iV. The X-operators are in general no longer 
projection operators but are assumed to obey still the commutation rules 

[xr,X^^]^^S,,{S,rXr±S.pXl'^) . (2) 

The upper (lower) signs in Eq.(2) hold for fermionlike (bosonlike or of mixed nature) X-operators. Per definition, 
fermionlike operators have the internal indices p = 0, g > or q = 0, p > 0, bosonlike ones p — q — or p, q > 0. 
Moreover, the diagonal X-operators are assumed to obey the constraint 

E ^r = y- (3) 

p=l...N 

In the usual case N = 2 both Eqs.(2) and (3) are fulfilled, and Eq.(3) is just the completeness relation. For a genecal 
N Eq.(3) means that at most N/2 electrons can occupy the N states at site i since X?" is a non-negative operatoicj. 
Moreover, it can be shown that the problem is completely specified if one assuines, in addition, that the diagonal 
operators Xf^ are projection operators for p > 0, exactly as in the case N — 2E3. Before proceeding we want to 
introduce the notation we will use in the following. The index 1 in X{1) denotes all the degrees of freedom (internal 

indices pi, qi, imaginary time ri, site index ii) of the X-operator, so that 1 = ( ^^^^ ) where 1 stands for 1 — (ii, ri). 

Following the Baym-Kadanoff formalism00'@, we define the single particle non-equilibrium Green's function in 
the presence of an external source K by 

G{12) = -< TSX{1)X{2) > / <S>, (4) 



S = Texp 



dlK{l)X{l) 



(5) 



In Eqs.(4) and (5) T is the time ordering operator and K^"^ the external source which is assumed to couple only to 
bosonlike operators X^''. J dl means dri where /? is the inverse temperature. It is convenient to introduce 

a normalized Green's function g such that Dyson's equation has the usual delta-function on the right side@@ 

d2{G^\l2) - S(12)).g(2l') = 5(1 - l'), (6) 
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where Go is given by 

G^\12) = 5{l - 2) A _ ^(1 _ 2)(i^"0(l)5,,,, - (1)), (7) 

and 

S(12) = - j d3w(132) < X(3) > + y d3d4d5u(134)5(45)7(52; 3) . (8) 
The exphcit expression for the function v is 

W(123) = 5{t2 - Ti)(5(t3 - n) [(till., + Ji^^^/25i^i^){5q^^)5q^o{l ~ Sp^o)Sp^pJq^p^ 



— SpiO^psoi^ - Sq3o)^p2q3^qiqi) + (^ilis + (-^^1*2/2 " ^ii ^2 )'^ili3 ) 
('5gi0'5g30(l — ^p3Q)^P20^q2qi^P3Pl ~ ^piQ^psoi^ ~ ^q3<i)^P2Pl^q2<i^q3qi)\ ■ (9) 

The square bracket on the right-hand side of Eq.(|^) contains two contributions: The first one describes the hopping 
of an hole between nearest neighbors with a spin-flip, the second one without a spin-flip. 

The mean field approximation is usually defined as an approximation for the self-energy E in which S contains all 
skeleton diagrams which are at most linear in g. This means in our case that the vertex 7 should be approximated 
by the contribution of zeroth order in (76^. Such an approximation, however, would violate Luttinger's theorem for 
every iV. Moreover, the relation between the number of particles at a site and the chemical potential would be not 
unique and depend on the way one calculates it. One way to get rid of these artefacts is to keep in the normal part 
of S only those terms which also are present in the large- limit and then to put N = 2. Another justification for 
such a procedure is the following: The leading normal and anomalous parts of S in Eq.(^) are of 0(1) and 0{1/N), 
respectively, in the 1 /N expansion. The calculation of the superconducting transition temperature involves in leading 
order only normal self-energies of 0(1), which, however, may already describe modulated phases. Calculations based 
on 1/iV expansions thus suggest that first one should solve the 0(1) normal state problem in leading order of the 
1/A^ expansion which may already imply the consideration of modulated phases. In a second step the solution is then 
used to solve the superconducting part in leading order in It has been found that the mean-field expression 

for the anomalous S is in general quite a good approximation for the total 0{1/N) contribution to the anomalous 
self-energy. Taking this for granted one arrives exactly at the above procedure to calculate E. 

Before working out the details of the above approximation for E it is useful to summarize the 0(1) results for S 
and g in the normal, unmodulated phase. In leading order of the expansion the vertex 7 in the self-energy can 
be approximated by the bare one, i.e., by 

7(12; 3) = (5(1 - 2)S{2 - ^)[Sp^o^p2aSp^qJq^q^ ~ Sq^oSq2aSq^p^dp^p^] . (10) 

Also in leading order of the 1/A^ expansion the expectation value of bosonic operators can be calculated in the absence 
of source field via 

<X^'^il)>^6pqgi^-^f^). (11) 

Inserting Eq.(p^ into Eq.(|8|) the self-energy is frequency- independent for zero source fields. Denoting it after a Fourier 
transform by e(k) we obtain for g 

5(k, ^^„) = . —— , (12) 

tuJn — e(k) — A + fi 



with 



e(k)-,5eo(k)+a(k). (13) 



eo(k) is the free electron dispersion eo(k) = — t(k)/2, and the doping (5, the function a, and the shift A are determined 
by the following expressions: 



i-s =—yf 



T 



(14) 



3 



(15) 



«(k) = -^5]j(k + p)r7(p). (16) 
p 

Nc is the total number of sites, f{x) the Fermi function f{x) — l/(exp(a;) + 1), and ?7(k) = /[e(k)/T]. J(q) is defined 
by >/(q) = 2J(cos(q2:) + cos(gy)), where J is the couphng constant Jyr-lpr nearest neighbor The long-range 
Coulomb interaction Vij, written in the Fourier space, has the expression^: 

^(q) = — , , (17) 



with A{qxjqy) — c[cos(ga;) -l-cos(gy) — 2] — 1. The constant c has been estimated to be around 50. According to Eq.([ri 
the long-range nature of V implies that limq^o ^(q) = cx3. In equilibrium this infinite large constant is compensated 
by the lattice due to charge neutrality. For a noncquilibrium value of the density this compensation no longer works. 
As a consequence, the isothermal compressibility, defined by k = [d / dn) , becomes 

K = ^0 + ^^"^^ > , (18) 

where kq means the compressibility without the Coulomb interaction. Whatever the value of kq is, the total k is 
according to Eq.(p^ always positive preventing any phase separation. 

III. COMMENSURATE AND INCOMMENSURATE 

FLUX PHASES 

The expressions for normal state quantities derived abpjje are valid as long as the doping is not too small, where 
the normal state is unstable with respect to other phasesE3. In partiatilar, the instability towards-bond-order states 
and flux phases has been investigated as function of the coupling J/<LrE3, and the temperature tI3l3. In these works, 
the new phases have been assumed to be commensurate with a commensurate modulation vector Qc = (7r,7r). (We 
use here the terms "commensurate" and "incommensurate" with respect to the lattice periodicity and not, as in Refs. 
|35| , ^ with respect to the electronic filling). On the other hand, numerical studiesE3t3 as well as a zero temperature 
slave boson calculationc3 indicated that an incommensurate flux state can be more stable than a commensurate one 
for a general doping. In this section we study this problem in more detail within our approach, especially, also at 
finite temperatures. 

The generalization of Dyson's equation for the Green's function ^(k, iujn) to the non-periodic case is 

(iw„ + fJ.)g{K q, it^n) - 4- X! - p, q - P, «w„) = NcS{q) , (19) 

p 

where we have defined the Fourier transformation by 

g(k, q, zc.„) = Y. 9ihJ, ^u;„)e^''<^^-''^^+^^■^^ . (20) 

hi 

Ri denotes the lattice vector to the site i. The instability towards an incommensurate state causes finite non- 
translational parts in E and g, which we write as: 

g{k, q, iujn) = g(k, iuJn)NcS{q) + Sg{k, q, ia;„) , (21) 

S(k, q) = E{k)N,S{q) + </.(k, q) . (22) 

In order to study the boundary of the incommensurate phase, it is sufficient to linearize Dyson's equation with respect 
to the non-translational parts, yielding 
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SgCk, q, iujn) = 9o{K itJ„)0(k, q)go(k - q, iw„) . 
From Eq.(^), calculated at large N^s, we obtain the relation between (j) and Sg: 

0(k, q) = ^ ^ B{k, q, p)T ^ ^^(p, q, ic^„)e'""0+ = 

p n 

= ^ X! ^' ■9o(P' *^»)ffo(p - q, iujn)<l>{p, q) 



p 71 

The kernel -S(k, q, p) can be written as a sum of separable kernels: 

J(q) J(k + p) 



where 



S(k,q,p) -t(k-q)+i(p) + V^(q) 



^F„(k,q)G„(p,q) 
a=l 



-F(k, q) = [tCk — q),l, J cos{kj.), J sin{kx), J cos{ky), J sm{ky)] , 



(23) 



(24) 



(25) 



(26) 



G(k,q) = 



l,i(k) + ^(q)- 



^(q) 



' cos(fc2;), — sin(fc;j^), — cos(fcj,), — sin(fcj,) 



The general linearized solution of the non-translational self-energy can then be written as 

0(k,q) =^/„(q)F„(k,q). 

a 

Inserting Eq. ( p8|) in Eq. (p^), we obtain the eigenvalue equation 

[Sat3 - aa/3(q)] //^(q) = , 



(27) 



(28) 



(29) 



where the matrix elements aQ,^(q) in Eq. ( P9[ ) are defined by 

aapiq) = -^^T^Ga(p,q)-F>(p,q)5o(p,iw„)go(p-q,«w„) 



(30) 



The boundary of the incommensurate phase is determined by the first onset of a non-trivial solution of the homogenetwis 
system Eq.(p9|). ^From a different point of view, such instabilities correspond to divergencies of the charge vertexr^lrl. 
The particular q^ where such divergencies occur ^determines the incommensurability vector. 

The undoped case has been studied in detailQ. Due to the Fermi surface topology, all instabilities occur for a 
commensurate wave vector, and the ground state can be described as a (tt, tt) flux phase with flux vr per plaquette. In 
our framework, the flux state is characterized by an order parameter Eq.(|2q) which is imaginary and the antisymmetric 
combination of its third and fourth component, i.e., 



bpLiK Qc) oc i[cos{kx) - cos{ky 



(31) 



Consequently, a d-wave gap is opened at half-filling of the band. Another state of interest which, however, has a 
higher energy at S — 0, is the socalled "kite" phase, which is twofold degenerate and described by the order parameter 
4>Ki{K Qc) oc i[sm{k^) ± sin(fcy)]. 

It has been argued thjatJac a finite doping both the flux plaquette and the instability or modulation vector qc are 
functions of the dopingOEjCJ. In order to investigate this question in more detail, we have studied instabilities of the 
normal state with respect to all possible order parameters Eq.(p^), which can be constructed from the eigenvectors 
of the 6x6 matrix in Eq. ([2S|). We found that the instability vector qc which can be restricted to the irreducible 
Brillouin zone corresponding to 1/8 of the total Brillouin zone always lies in the (0, 7r)-(7r, tt) direction, going smoothly 
to (tt, tt) for zero doping. Fig. 1 shows the calculated phase diagram in the S ~ J plane at zero temperature. From 
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FIG. 1. Solid line: Zero-temperature boundary between normal state N and incommensurate flux state FL. The arrow 
marks a transition between an incommensurate flux at low and an incommensurate kite phase at high dopings. Dashed lines: 
Boundaries between normal state N and a commensurate flux or kite state. 



now on we put t = 1 so that all energies like 3 are measured in units of t. The dashed lines describe transitions 
to commensurate states disregarding competing incommensurate states. For J < 1 there is a transition between the 
normal state at large dopings to a commensurate flux state at low dopings. For J > I the normal state is with 
decreasing doping first unstable with respect to the kite phase and the kite phase then with respect to a flux state. 
Allowing also for incommensurate states the solid line in Fig. I shows the boundary of the normal state with respect 
to an incommensurate flux state at J < 0.5 and the kite phase at J > 0.5 where the exact transition point between 
the two incommensurate states have been marked by an arrow. For J > 0.5 there is probably another solid line 
describing a transition between an incommensurate kite and flux phase similar as in the commensurate case. Since we 
will confine ourselves in the following to the parameter range J < 0.5 relevant for high- Tc superconductors we have 
not tried to calculate this additional phase boundary. 

Fig. 2a) characterizes the components of the order parameter along the incommensurate boundary of Fig. f . The 
labels 3,4,5,6 correspond to a = 3,4,5,6 in defined in Eq.(p6|). For dopings below ~ 0.175 the order parameter 
is ~ cos(A;a;) — cos(fcy) and thus has d-wave or, using the proper point group classification. Fa symmetry. Strictly 
speaking, <^ has also a small additional term oc sin(fcy) which may occur in incommensurate but not in commensurate 
states. For dopings above 6 ^ 0.175 we have </) cx smik^^ and thus an order parameter with Fs symmetry. Writing the 
instability vector as qc = (1, x)'k the dependence of the value x on the doping is shown in Fig. 2b). qc approaches the 
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**♦, (b) 
J=0.3 ♦ *♦♦, . 



doping 8 

FIG. 2. (a) Evolution of the eigenvector components (3,4,5,6 correspond to cct&{kx\ sin(fci,), cos(fcy), sin(fcy), respectively), 
along the incommensurate boundary in Fig. 1; (b) Corresponding dependence of the instability vector qc = (l,a::)7r. 



commensurate wave vector Qc at zero doping. With increasing doping it moves slowly away from Qc and exhibits a 
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FIG. 3. Incommensurate (solid line) and commensurate (dashed line) phase boundaries in the T ~5 plane. Insert: Evolution 
of the instability vector qc = (1, x)-n as function of T. 



jump at (5 ~ 0.175 where the flux state is replaced by the kite state. For the experimental value J/t ^ 0.3 only the 
flux phase is possible and is the stable phase for dopings below 5 ^ 0.13. 

Fig. 3 shows the phase diagram in the T — 5 plane for J = 0.3. The solid line describes again the incommensurate, 
the broken line the commensurate state. The inset of that Figure shows the temperature dependence of the quantity 
X of the instability vector qc = (1, x)tt. For temperatures larger than ^ 0.014 qc coincides with Qc so that the broken 
and solid lines become identical. This behavior is caused by the thermal width of the Fermi function: though the 
gap in the one-particle density opens not exactly at the chemical potential the thermal smearing around the Fermi 
surface is large enough to lower the free energy by taking advantage of the high density of states corresponding to 
the half-filled case. In the following we will assume that the flux state is commensurate. This assumption simplifies 
considerably calculations which also take superconducting states into account. On the other hand Fig. 3 suggests 
that this is not an unreasonable approximation. 



IV. COMPETITION BETWEEN SUPERCONDUCTIVITY 

AND FLUX PHASES 



In this section we consider instabilities of the normal and of the flux state with respect to superconductivity. 
Superconducting instabilities occur at most in 0(1/ N) of our 1/iV expansion. This means that it is sufficient to 
calculate the Green's functions entering the linearized gap equation for superconductivity in the leading order 0(1). 
A detailed study of the normal to superconducting transition showed that the usual mean field term dominated all 
the other contributions to the kernel of the gap equation. So we will keep just this term and neglect all the other 
terms which are also of of 0{1/N). Moreover, it was shown that the leading superconducting instability occurs in 
the d-wave-like T3 channel so we will consider only a d-wave order parameter. With respect to the order parameter 
matrix cf) we will keep the component oc cos{kx) — cos{ky) discussed in the last section. The usual charge-density 
wave order parameter is according to Eq.(p8|) connected to the columns 1 and 2 and the sum of 3 and 5 and will be 
included in the next section. Finally, the components cx sin(fca;) and cx sin(fcj,) are associated with the kite phase and 
do not play a role for J < 0.5 as shown in the previous section. 

In the presence of a commensurate fiux phase and singlet superconductivity the operators of the following row 
vector 

^(k) = (XO-(k),X^"(-k),XO'^(k-Q,),X'*°(k-Q,)), (32) 

are coupled in the Green's functions. We thus define a 4 x 4 matrix Green's function 

G{k) = - < TStp\k)i}{k) > / <S>, (33) 

where ip^ is the hermitian conjugate of ip, i.e., a column vector, k in Eq.(^3[) denotes the four-component vector 
k = (k, iwn)- We also note that in the absence of magnetic ground states the Green's functions G are spin-independent 
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which allows to drop all spin indices. Going then over to normalized Green's functions we write the 4x4 matrix g(k) 
as 



9{k) = 



/ 9ii{k) gi2{k) 9iz{k) 
92i{k) g22{k) 524(fc) 

gisik - Qc) 5ii(fc-(?c) 9i2{k - qc) 

\ g2i{k-qc) g2i{k~qc) g22{k ~ qc) 



(34) 



Here we used the fact that the matrix elements 5i4, 523, <?32, 54i of g must vanish and the explicite expression ( |3^ ) to 
connect different matrix elements. We also used the four-component vector q^ — (Qc,0). Dyson's equation becomes 
a 4 X 4 matrix equation 



iiOnl - S(fc) • g{k) = I . 



The self-energy matrix S has the general form 
S(k) 



/ I]n(k) Ai2(k) i</.i3(k) 

A2l(k) S22(k) Z(/.24(k) 

z0i3(k-QJ En(k-QJ Ai2(k-QJ 

i024(k-QJ A2i(k-QJ S22(k-QJ 



V 

Explicit expressions for the elements of E are obtained from Eq.(H): 

I],,(k) = {-iy+\l - n) eo(k) - a,,(k) + A,, - ^ , 

with 



(35) 



(36) 



(37) 



(38) 



i(p)-^^ + nq-0) 



r^g,,(p,*c.„)e^""°\ 



(39) 



and 



Ay(k) = 5][J(k + p) - y,(k + p)]T^5,,(p,zc.„)e-""' 



(40) 



(41) 



*0y(k) = -^5] J(k + p)r^g,,(p,zc.„)e^""''^ , (42) 

p n 

where Vd(k) = 2Kir! [cos(fc^) -I- cos(fct,)] is the nearest neighbor part of the Coulomb interaction. The wave vector 
k is either k or k — Qc, so that Eqs.(37) - ( |4^ ) determine all matrix elements of the self-energy. In particular, all 
elements of S are independent of frequency in agreement with the notation in Eq.(|3^). Eqs.(|3^) and (^6|) determine 
all elements of g and S. It is easy to see that many of them are actually not independent. Writing for the diagonal 
elements Y^u (k) = ta (k) + Xa — ^ we have A = Xu and 

e(k) = eii(k) = -e22(k) - -en(k - QJ - e22(k - QJ . (43) 

Similarly, one obtains for the elements of the order parameters 

A(k) = Ai2(k) = A2i(k) = -Ai2(k - QJ = -A2i(k - QJ , (44) 
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0(k) = </)i3(k) = 024 (k) = -013 (k - QJ 



(45) 



In Eq.(|44|) we have used the fact that A can be chosen to be reaL 

I](k) can easily be diagonahzed and one obtains four branches for the excitation spectrum: 



±i?±(k) = ±v/K(k) ± (a* - A)P + A(k)2 



with 



Eq.(p5[) can therefore easily be inverted and the sum over frequencies in Eqs.(^ 
following set of self-consistent equations: 



n = 1 



2 U{p)->^ + ^^ 



E' 



tanh 



E+ip) 



2T 



e(p) + X-fi 



tanh 



E-{P) 



2T 



(46) 



(47) 

3) carried out. We obtain the 



(48) 



J(0) 



n , 



(49) 



(l^) = -^E''^(k + p)r;(p), 



(50) 



^(^) = W' E '[-^(^ + p)-VAk + p)] (P) 



(51) 



'^(^) = ^E''^(k + p)774p), 



(52) 



where 



7?(k) = - 



ejk) r g(k)-A + ^ 

e(k)l 2£;+(k) 



£;+(k) 



2r 



^(k) + A - 

2£;_(k) 



tanh 



i?_(k) 



2T 



(53) 



N i ^(k) 



^^+(k) 



2r 



A(k) 
2£;_(k) 



tanh 



E-ik) 



2T 



(54) 



0(k) f e(k)-A + M , , 

'^^('^^ ^ ?(k) I 2i?+(k) '""^ 



i?+(k) 



2T 



e(k) + A - 
2i;_(k) 



tanh 



i^-(k) 



2r 



(55) 



and the prime on the summation indicates that the sum is restricted to the reduced Brillouin zone. 

Eqs. (^8|)-(^2|), together with Eqs. (^)-(|55|), determine in a self-consistent way all the properties of the system. 
Moreover, it is possible to construct the thermodynamical potential r2(/i,T) as that function which satisfies the 
extremal conditions 



5D. 
(5a(k) 



0. 



5n 
<5A(k) 



0, 



5^ 



0, 



(56) 



together with the conditions dQ./d\ — 0, —dfl/dfi — n. Performing a Legendre transformation on Vl one obtains the 
following expression for the free energy F{N,T): 
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FIG. 4. Phase diagram of the t-J model for decoupled d-wave superconducting (SC) and commensurate flux (FL) order 
parameters. 



F(7V,r)-(^-A)(n-l) 



2 cosh 



2T 



In 



2 cosh 



2T 



27V2 



k,p 



+ ^ E '["^(k + p) - y,(k + p)]r;A(k)77A(p) 



(57) 



k,p 



After having derived the system of equations for the order parameters we are going to analyze the phase diagram 
of the t — J model within our approach. We choose the generally accepted value J — 0.3 using always t as the energy 
unit. An estimate of the Coulomb repulsion between nearest neighbor sites, Km, may be more controversial. It seems 
reasonable to assume that Vnn is of the same order as J, so we have chosen the value Vnn = 0.5 J. This particular 
choice means that the contribution from density-density interactions in the J and Vnn terms cancel each other in the 
d-wave superconducting channel, so that superconductivity is driven only by spin-exchangeEil. The role played by Vnn 
and the Coulomb interaction in the phase diagram will be discussed in more detail in the next section. 

Fig. ^ shows the instability line of the normal state with respect to d-wave superconductivity (solid line) and 
commensurate flux phase (dashed line) in the T — 6 plane assuming that the two phases are uncoupled. Tc of the 
uncoupled superconducting phase increases with decreasing doping 6. This behavior is caused mainly by an increase 
of the density of states with decreasing S due to the narrowing of the band and the decreasing distance of the Fermi 
energy from the Van Hove singularity at the middle of the band. The transition temperature Tp^ to the flux phase 
becomes non-zero at around S ~ 0.13 and assumes very rapidly large value towards lower dopings. Though this 
instability is caused by nesting properties of the quasi two-dimensional Fermi surface the dashed line in Fig. ^ 
indicates that it is much stronger than the instability towards superconductivity. Both phases have order parameters 
of d-wave symmetry, i.e., nodes along the [1,1]- and maximum absolute values along the [1,0]- and [O,l]-directions in 
k-space. Both phases thus try to reconstruct the Fermi surface mainly around the X— and Y— points creating there 
a gap in the single-particle excitation spectrum. In the case of superconductivity the gap opens always right at the 
Fermi energy and moves with doping. In contrast to that the gap of the flux phase is fixed at the middle of the gap 
due to the assumed commensurability. Nevertheless, there will be a large competition between the two phases for not 
too large dopings. Fig. ^ suggests that the flux phase is able to reconstruct also electronic states further away from 
the Fermi surface compared to the superconducting phase which affects mainly electronic states close to the Fermi 
energy. 

Fig. 5 shows the phase diagram of the t — J model if the interaction between the flux and superconducting phases 
is taken into account. For 6 >^ 0.13 only the superconducting phase is stable at low temperatures. At around 
6 = 0.13 the flux phase order parameter becomes non-zero in the superconducting phase and a coexistence region 
exists of superconductivity and flux phase. Because the two order parameters have the same symmetry and aim to 
reconstruct the same parts of the Fermi surface the stronger of the two phases tries to suppress the weaker one. Since 
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FIG. 5. Phase diagram of the t-J model taking into account the competition between superconductivity and commensurate 
flux phase. 

TpL > Tc for 6 <^ 0.13 the superconducting phase is rapidly suppressed for decreasing dopings. As a resuh the 
solid line in Fig. ^ which increases monotonously with decreasing doping tends rapidly to zero below S ^ 0.13 due 
to the interaction with the flux phase. The maximum value for Tc coincides rather accurately with the onset of the 
flux phase at T = 0. Fig. ^ shows the flux per plaquette <& in the interacting case as a function of doping for three 
different temperatures. The flux always assumes its maximum value of tt at zero doping. With increasing doping or 
with increasing temperature $ decays rather fast. In the coexistence region of superconductivity and flux phase $ is 
small but non-zero. 

Both Figs. ||and|| show that at low temperatures T <~ 0.01 an incommensurate flux state is more stable than the 
commensurate one. In an incommensurate flux state the one-particle gap opens like in the case of superconductivity 
right at the Fermi energy. From this one might expect an even larger competition between flux and superconductivity 
phases and a more rapid quenching of Tc at low dopings than in the case of a commensurate flux phase. We thus 
arrive in a natural way at a scenario with a quantum critical point. Omitting superconductivity the metallic state 
at large dopings passes at zero temperature with decreasing doping through a critical point into a non-metallic flux 
state characterized by an order parameter with d-wave symmetry and an incommensurate modulation vector. At 
finite temperatures the long-range order of the flux state is destroyed but there are still regions of small and large 
fluctuations in the flux order parameter. Allowing also for superconductivity the surroundings of the quantum critical 
point becomes superconducting with a Tc which has a maximum at the critical point and decays rapidly at larger or 
smaller dopings. One attractive feature of our phase diagram is the near coincidence of critical point and maximal Tc 
which is a necessary consequence of the competition of flux and superconducting phases. This coincidence seems to 
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FIG. 6. The flux <E> per plaquette as a function of doping for different temperatures. 
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be a generic feature of high- Tc oxides and does not rely in our approach on parameter choices or fine-tuning. 



V. COULOMB INTERACTION AND INCOMMENSURATE 

CHARGE-DENSITY- WAVE 

We have seen in section II that macroscopic phase separation is impossible in the presence of long-range Coulomb 
forces. However, it is known that in such a situation the charge instability at q = earresponding to phase sep^, 
ration is shifted to a finite q, leading to an incommenspjate charge-density-wave stateEa. Commensurate (CDW)t3 
or incommensurate charge-density-wave states (ICDW)E£la have been proposed to account for the metal-insulator 
transition at T = 0, the quantum critical point, and the pseudo-gap features at finite temperatures in the underdoped 
regime of high- Tc oxides. Though we have proposed a different candidate for the insulating state in the previous 
section the t — J model may exhibit charge separation and ICDW's in addition to the discussed flux phase. Thus 
the phase digram in Fig. 5 may have to be modified. We therefore study in the following possible CDW instabilities 
including in the Hamiltonian also the long-range Coulomb interaction V{q) defined in Eg. (]l7|) . For this we consider 
a general ground state characterized in general by nonvanishing order parameters for d-wave superconductivity and 
a commensurate d-wave flux state. The corresponding Green's function and self-energies form 4x4 matrices as has 
been discussed in section IV. Atop on this ground state we allow for a small ICDW and check whether its amplitude 
can be nonzero. The procedure is similar to that used in section III for an incommensurate flux state. 

We write the Green's function and the self-energy as 

g(k, q, iujn) = 9{K iuJn)NcS{q) + Sg{k, q, iujn) , (58) 
S(k, q) = tCk)N,S{q) + 4>CDW{K q) . (59) 



The first term on the right-hand sides of Eqs.(|52) and (g9|) describe the state with superconductivity and commensurate 
fiux phase, both with d-wave symmetry. (j)cDW and Sg are small non-translational additions to S and g due to the 
ICDW. It is sufficient to linearize Dyson's equation with respect to the non-translational parts yielding 

5g{k, q, iw„) = g{k, iuj^) ■ (i>CDw{^, q) • 3(k - q, iuJn) ■ (60) 

Since we are dealing with a CDW instability (j)cDwO^, q) is diagonal in the 4x4 space and can be written as a linear 
combination of the two 4x4 matrices: 

where a 3 is the usual third Pauli matrix. Explicit calculations of the diagonal elements of the self-energy using Eq. (H) 
yield, similar as in Eq.(p4), the result 



icDwiK q) = T3t{k - q)^ 5^ '7^5^ Tr [fo • 5g{p, q, iw„)] 

p n 

E '^(P)^ E [^3 • Sgip, q, zc.„)] 



-Tn- 

P 



-M-^ + ^(q))^ E '^E [t3 • Sg{p, q, zc.„)] 

*^ p n 



^3:7^ E '-^(^ + P)^E [^3 • Sg{p, q, zc.„)] . (63) 
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FIG. 7. Phase diagram of the t-J model, considering also an incommensurate charge-density- wave (ICDW). The dashed line 
between ICDW and ICDW4-FL is purely indicative. 



Eq.(|63[) can be written in the more compact form 

^CDw(k,q) = ^^'r^^F„(k,q)Tr[G„(p,q).5g(p,q„ 

p 71 a. 

with the abbreviations 

^(k,q) 



lUJn 



t(k - q)T3,ro, — 



G(k,q) 



fo,t(k)T3 



^(q) 



^(q) 



To, - 



-T3 



Similar as in section III, the solution of the ICDW order parameter can be written as 

<^Ci5H'(k,q) = ^/a(q)/'a(k,q) . 

Using Eq.(^^ the expansion coefhcients satisfy the following system of 3 equations 

P 

where the matrix elements aQ./3(q) are defined by 

aa/3(q) = Jf^'^^^"^ [Ga(p,q) •go(p,»t^«) •F^(p,q) • go(p - q, 



(64) 



(65) 



(66) 



(67) 



(68) 



(69) 



As usual the phase boundary is determined by the onset of the first non-trivial solution of the homogeneous system 
Eq. (|68|) . The particular qc where this occurs determines the incommensurability vector of the ICDW. 

Taking also ICDW's into account the calculated phase diagram is shown in Fig. 7 using J — 0.3 and Km/ J = 0.5. 
We also used the long-range Coulomb potential Eq.([l7|) in the calculation. We characterize its strength by its value 
Vnn between nearest-neighbor sites. Fig. 7 should be compared with Fig. 5 where the same parameter values have 
been used but the Coulomb interaction was confined to nearest neighbors. The curves for T^, and Tp^ are very similar 
in both cases. The region of the flux phase is now, however, split into two regions. In one region at larger dopings 
only the flux phase is stable; in the other one at lower dopings the flux and the incommensurate charge density phase 
coexist with each other. At high temperatures the pure ICDW is stable at lower dopings. The Figure indicates that 
the ICDW has no important influence on the Tc curve and, in particular, to its maximum value at optimal doping. 
The latter is still determined solely by the instability towards the flux phase at around i5 = 0.13. 
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FIG. 8. Evolution of the phase diagram for J = 0.3 and different Coulomb interaction strengths, characterized by the 
nearest- neighbor constant Vnn- For Vnn = the ICDW instability occurs at qc = corresponding to the divergence of the 
isothermal compressibility and the onset of phase separation (dot-dashed line). 



Finally we discuss the dependence of the phase diagram on the Coulomb repulsion strength Vc, defined in Eq. 
Similar as in Fig. 7 we use instead of Yc the value T^im i-e., the Coulomb potential between nearest neighbor sites, to 
characterize the strength of the Coulomb potential. Changing y„„ we can distinguish between two extreme limits. In 
the case of negligible Coulomb repulsion, i.e. Ki„ = 0, the attractive charge-charge term of the t — J model becomes 
important. As a result the superconducting region becomes large and wipes out the flux phase as shown in the upper 
panel of Fig. 8. Lowering the temperature from high values one crosses the solid line and enters the superconducting 
region. The phase boundary between normal and flux phase in Fig. 5 lies now within the superconducting region 
where according to the calculation this boundary no longer exists. This can be understood from the fact that the 
superconductivity order parameter has already reconstructed the Fermi surface, especially near the points X and F, so 
that an additional order parameter with d-wave symmetry cannot lower further the free energy. At zero temperature 
the ground state is always superconducting in agreement with the arguments of Ref. ^ Exactly at zero doping the 
superconducting and the flux phases become equivalent, again in agreement with previous arguments. The isothermal 
compressibility diverges along the dot-dashed line in the upper panel of Fig. 8. This means that on the left side of 
this line the normal and superconducting states are unstable as homogeneous phases, and the system separates into 
two phases, one with zero and the other with a high hole density. 

Increasing Vnn from zero to finite values the superconducting state is more and more suppressed whereas the 
boundary to the flux phase is unaffected. This is shown in the middle panel of Fig. 8 for Vnn I J = 0.5 which is 
the same diagram as Fig. 7. Moreover the superconducting region splits up into a pure superconducting part at 
larger dopings and a region at lower dopings where the superconducting and the flux order parameters coexist. When 
Vnn exceeds the value J the total effective hole-hole interaction becomes repulsive and superconductivity is totally 
suppressed. At the same time the instability of the flux phase with respect to an additional ICDW moves towards 
smaller dopings, i.e., the region of the pure flux phase increases also on the cost of the coexistence region of flux and 
ICDW phases. This is illustrated in the lower panel in Fig. 8. Fig. 8 demonstrates, in particular, two things: with 
increasing Vnn the CDW instabilities move monotonically to lower dopings. The position of these instabilities is in 
general far away from optimal doping and thus does not influence much the region where superconductivity is the 
stable phase. For Vnn > optimal doping is more or less determined by the onset of the flux phase at T = and thus 
tied to this instability. 

We are now in a position to make a comparison of our results with those of other treatments. Ref. ^ also 
enforces the constraint by X-operators but does not find any instability of the normal state with respect to a flux 
phase. Considering the case N — 2 from the outset Ref. ^ uses a mean-field like decoupling procedure which 
violates Luttinger's theorem. For instance, the Fermi surface for 6 — 1/3 corresponds to half-filling in theories where 
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Luttinger's theorem is fulfilled. The absence of a flux phase instability of the normal state as well as a finite Tc for 
superconductivity even at J = may be artefacts due this short-coming. Ref. calculates the expectation values 
of bosonic Hubbard operators and the Green's functions with different perturbation expansions finding also solutions 
for the 71 (/i) relation which satisfy Luttinger's theorem. The resulting Tc{5) curves are similar to ours but again flux 
phases and their competition with superconductivity are missing. A basic inconsisten cy p roblem inherent in simple 
decoupling schemes with X-operators can also be inferred from a comparison of Ref. ^ with Ref. Calculating 
expectation values of bosonic Hubbard operators from Green's functions using the projection properties of Hubbard 
operators or from thermodynamic relations yields different n{p) relations. In contrast to that the expansion 
yields a unique n(/z) relation and also satisfies Luttinger's theorem. On the other hand we cannot say much about the 
conveijgence of the expansion in general. However, in the case of the density fluctuation spectrum it has been 
shownEZi that the leading order in 1 /N can already account for most features found in the spectra calculated by exact 
diagonalizations for N = 2 for small systems and that the remaining discrepancies nearly vanish if next-to-leading 
contributions are also taken into accountc2l. 

Finally we compare our results with treatments where the constraint is enforced using slave particles. One general 
result of these approaches isE3E3 that the staggered flux phase is always unstable at T = against d-wave supercon- 
ductivity. This agrees with our findings, see the upper panel of Fig. 8. Fig. 4 of Ref. || and Fig. 1 of Ref. ^, if scaled 
to our value J/t — 0.3, show at finite temperatures no or only a very small region at very small dopings where the 
flux phase is stable. In our case the flux phase is wiped out either by superconductivity or by phase separation as 
shown in the upper panel of Fig. 8. Taking also Coulomb interactions into account to prevent macroscopic phase 
separation we find that the flux phase becomes stable above the superconducting phase yielding a maximal Tc of 
5 ~ 0.12. This value is much larger than the value 5 ~ 0.03 obtained in Ref. || and also closer to the experimental 
one of (5 0.15. The decrease of with decreasing 6 in the underdoped region is determined in Refs. by the 
condensation temperature of the slave boson particles. Such a Bose condensation does not exist in our approach. 
Instead the decrease of Tc in the underdoped region is caused in the present approach by the competition of the two 
d-wave order parameters describing the flux and the superconductivity phase. We also note that experimental data, 
have been interpreted in a phenomenological way as a competition of the superconducting and an unknown phaseE3 
and the resulting phase diagram is very similar to those of Figs. 5 and 8. The leading order of the 1/iV expansion 
is certainly insufficient for a proper description of the undoped case ^ = 0. It is now generally accepted that the 
obtained resonance-valence bond instability is in this case somewhat weaker than the instability towards long-range 
antiferromagnetism. The latter instability, however, can only be obtained by taking higher order contributions of the 
1/A^ expansion into account. 



VI. CONCLUSIONS 



In this paper we have derived the phase diagram of a generalized t — J model taking superconducting, flux, and 
charge density wave states into account. The investigation was based on the leading expressions of a 1/A'^ expansion 
enforcing the constraints by means of X-operators. We found a strong competition between d-wave superconducting 
and d-wave flux states. As a result the transition temperate Tc for superconductivity showed a maximum near a 
doping value 5 = (5c ~ 0.13 for J/t = 0.3. This value is determined essentially by the onset of an (incommensurate) 
flux phase at 5c- To simplify the calculations we assumed the flux phase to be commensurate. We showed that this is 
correct for T /t > 0.01. Below this temperature the flux phase is incommensurate which, if taken into account, would 
presumably not change substantially our conclusions. We also studied the influence of long-range Coulomb forces on 
the phase diagram. As a result incommensurate charge density waves become stable or coexist with the flux phase at 
lower dopings well separated from the superconducting region influencing the latter at most in a marginal way. 

Our results can be interpreted in terms of a quantum critical point scenario. Disregarding superconductivity the 
metallic state at large dopings 5 > 5c passes to a non-metallic, incommensurate flux state with d-wave symmetry for 
5 < 5c- Allowing also for superconductivity Tc increases from the over- and underdoped sides and shows a maximum 
around 5c- On the underdoped side the superconducting phase coexist with the flux phase up to Tc where a pure 
flux state becomes the most stable state up to the normal state at high temperatures. Two different proposals for 
the non-metallic state in the quantum critipaLx)oint scenario have been made, namely an antiferromagnetida and an 
incommensurate charge density wave stateOo. In comparison with these states we would like to point out three 
attractive features of our proposal for the non-metallic state. The instability towards an incommensurate flux phase 
is a generic feature of the t — J model and is also present if second-nearest neighbor hopping or Coulomb forces are 
additionally taken into account. For J/t < 0.5 the flux phase has d-wave symmetry, i.e., the same symmetry as the 
most stable superconducting state. Since the flux phase instability is much stronger than the superconducting one Tc 
is heavily suppressed by the flux phase in the underdoped regime. As a necessary consequence the maximum value 
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for Tc lies near the onset of the flux phase at 5c- 
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